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Classwide Extensions of Vocabulary 
Intervention Improve Learning of 
Academic Vocabulary by Preschoolers 


Yagmur Seven,? Katharine Hull,? Keri Madsen,* John Ferron,* Lindsey Peters-Sanders,” 
Xigrid Soto,? Elizabeth S. Kelley,” and Howard Goldstein? 


Purpose: Many preschoolers, especially those from low- 
income households, would benefit from instruction to 
enrich their vocabulary and language repertoires. Yet, 
explicit instruction of vocabulary and language skills 
generally occurs infrequently in early childhood education 
settings. This study investigated the additive effects of 
teacher-led, classwide review strategies to a previously 
studied small-group intervention on children’s learning of 
academic vocabulary. 

Method: Participants included 23 children with limited oral 
language skills at risk for reading difficulties enrolled in 
single-case experimental designs. Effects of the classroom 
strategies alone also were examined in 10 children with 
above-average language abilities from 2 classrooms. 
Results: Visual analyses of the adapted alternating 
treatments designs showed consistent learning 
improvements when vocabulary instruction was extended 
into the classroom for 12 children, ceiling effects were 
evident for 3 participants regardless of condition, and 


inconsistent or minimal effects were demonstrated by 

8 participants. Multilevel modeling used to evaluate the 
effects statistically revealed strong treatment effects. In 
addition, the 10 children with above-average language 
showed impressive learning of vocabulary words from 
books subject to teacher review strategies in comparison 
to words from books to which they were not exposed. 
Teachers varied in the extent to which they implemented 
review strategies in their classrooms. Nevertheless, their 
responses to social validity assessments were positive, 
supporting the feasibility of this intervention. 
Conclusions: The addition of classwide review and 
practice opportunities is an effective means of 
enhancing the effects of an easy-to-implement small- 
group intervention that teaches challenging vocabulary 
words within prerecorded stories. This approach holds 
promise as a way to shrink the pervasive word gap that 
typically exists when children in high-poverty communities 
enter school. 


evelopment of oral language is critical during 
early childhood, providing a foundation for liter- 
acy and nearly all other academic performance 


(Roth, Speece, & Cooper, 2002; Walker, Greenwood, 
Hart, & Carta, 1994). Specifically, oral language is known 


to be a reliable predictor of reading proficiency with a clear 


relation emerging between early vocabulary knowledge and 
later reading comprehension (Kendeou, van den Broek, 
White, & Lynch, 2009; Quinn, Wagner, Petscher, & Lopez, 
2015; Roth et al., 2002). The consequences of poorly devel- 
oped vocabulary can be devastating as children who start 
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school with limited vocabulary are prone to developing 
literacy difficulties that persist throughout school (Hart 
& Risley, 1995; Hoff, 2013; National Research Council, 
1998). Unfortunately, children exhibit wide differences in 
vocabulary knowledge upon school entry, placing many 
children at risk for academic disadvantage (National 
Research Council, 1998; Walker et al., 1994). Because 
the stakes are high for children with limited vocabulary 
knowledge, efforts to improve oral language and vocabu- 
lary in at-risk preschool children are paramount. 
Meta-analyses have shown that vocabulary instruc- 
tion for young children, often designed to be delivered 
within the context of storybook reading, yields mostly 
moderate effects (Marulis & Neuman, 2010; Mol, Bus, & 
de Jong, 2009; National Early Literacy Panel, 2008). Various 
approaches are used to promote vocabulary skills within 
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the context of storybooks. For example, interactive 
reading and dialogic reading (Lonigan & Whitehurst, 1998; 
Whitehurst et al., 1988, 1994) are designed to develop oral 
language in young children through the use of purposeful 
adult—child interactions and scaffolding during storybook 
reading. Although these approaches to oral language 
development are widely studied and commonly used, the 
meta-analysis by Mol et al. (2009) concludes that these 
approaches are only moderately effective when imple- 
mented by researchers and far less effective when imple- 
mented by trained teachers. Marulis and Neuman examine 
a broader spectrum of approaches to vocabulary instruction 
in their meta-analysis of studies with preschoolers and 
kindergarteners. In contrast to Mol et al., these researchers 
specifically examined the effects of explicit instruction 
on vocabulary knowledge. Their findings suggest that 
approaches that explicitly teach words and meanings and 
provide multiple examples before, during, or after story- 
book reading are most effective at improving word learning. 
Multiple studies examining the use of explicit vocabulary 
instruction within the context of storybooks further dem- 
onstrate the power of this instruction when provided to 
young children across ability levels (Beck & McKeown, 
2007; Coyne, McCoach, & Kapp, 2007; Goldstein et al., 
2016; Justice, Meier, & Walpole, 2005; Loftus-Rattan, 
Mitchell, & Coyne, 2016; Pollard-Durodola et al., 2011). 
Even small doses of carefully planned, explicit instruction 
have significantly improved oral language and vocabulary 
outcomes for at-risk children in preschool and kindergarten 
(Marulis & Neuman, 2010). 

Despite the evidence supporting explicit vocabulary 
instruction, there is growing concern that preschool children 
receive little or no oral language instruction during the 
school day (Greenwood et al., 2013; National Early Literacy 
Panel, 2008). Observational studies of instruction in early 
childhood settings reveal that many preschool teachers fail 
to provide high-quality oral language instruction to chil- 
dren in need (Dickinson, 2011; Justice, Mashburn, Hamre, 
& Pianta, 2008; Wright, 2012). One approach to increasing 
the delivery of high-quality oral language instruction in 
early childhood classrooms involves changing teachers’ 
practices (Cabell, Justice, McGinty, DeCoster, & Forston, 
2015; Cabell et al., 2011; Wasik, Bond, & Hindman, 2006; 
Wasik & Hindman, 2014). For example, Wasik and her 
colleagues trained preschool teachers to improve their 
vocabulary instruction and later observed teachers during 
read-alouds to code the frequency of their discussions 
about vocabulary. These researchers found that students 
of teachers who discussed words more frequently demon- 
strated better vocabulary outcomes. Unfortunately, there 
still are many preschool teachers who find it challenging 
to provide high-quality oral language instruction despite 
access to training and instructional tools, as has been 
observed across early childhood settings (Dickinson, 2011; 
Justice et al., 2008; Pence, Justice, & Wiggins, 2008). Fidelity 
may be low for teachers implementing language curricula 
in preschools (e.g., Dickinson, 2011; Pence et al., 2008). 
In addition, even when teachers are able to achieve fidelity 


after being provided with a curriculum and professional 
development, there is evidence that they may continue to 
fall short in the quality of their instruction (Justice et al., 
2008). The challenges faced by preschool teachers are further 
complicated by a lack of early childhood curricula designed 
to help teachers deliver instruction that fosters vocabulary 
growth in children (Neuman & Dwyer, 2009). Thus, there 
is a need for well-designed, explicit vocabulary curricula 
that are highly effective yet easy for early childhood profes- 
sionals to implement with fidelity in preschool settings. 

Story Friends (Goldstein et al., 2016) is a supple- 
mental curriculum for preschool children that delivers ex- 
plicit vocabulary instruction through engaging, prerecorded 
storybooks, thereby making the curriculum inherently easy 
to implement with fidelity. Each audio book provides 
embedded instruction of four sophisticated words that is 
based on several key principles for robust vocabulary in- 
struction described by Beck, McKeown, and Kucan (2002, 
2008, 2013). In Story Friends, the target words are first 
presented within the narration of the story, and primary 
instruction of the target words immediately follows. The 
lessons provide a child-friendly definition and multiple 
examples using each word in a variety of contexts. Chil- 
dren are given multiple opportunities to practice and to 
actively respond during this instruction. In addition, chil- 
dren listen to each book three times, thereby receiving 
repeated exposure to each lesson. 

Prior studies have shown Story Friends to be effec- 
tive at increasing vocabulary knowledge in preschoolers at 
risk for language and literacy difficulties. On average, chil- 
dren in these studies demonstrate learning of 30%-50% of 
the instructed words (Goldstein et al., 2016; Greenwood 
et al., 2016; Kelley & Goldstein, 2014; Kelley, Goldstein, 
Spencer, & Sherman, 2015). This proportion of words 
learned compares favorably to other studies of explicit 
vocabulary intervention in which children learned a smaller 
percentage of words taught (e.g., Justice et al., 2005; Loftus, 
Coyne, McCoach, & Zipoli, 2010). Considering the rela- 
tively low dose of instruction (approximately 45 min per 
week) and the limited language abilities of participants, 
Story Friends offers an example of an effective vocabulary 
intervention that can be implemented in real-world class- 
rooms settings. 

Although Story Friends has consistently resulted in 
robust effects on word learning in young children, it is pos- 
sible that these effects have not been maximized. A grow- 
ing body of research has examined the effects of extending 
vocabulary instruction beyond storybook reading. These 
extensions, which provide increased opportunities for prac- 
tice throughout the day, have resulted in significantly 
better vocabulary learning outcomes (Beck & McKeown, 
2007; Coyne et al., 2007; Coyne, McCoach, Loftus, Zipoli, 
& Kapp, 2009; Loftus-Rattan et al., 2016; McKeown & 
Beck, 2014; Wasik & Bond, 2001). In these studies, young 
children who received extended instruction and practice of 
vocabulary beyond the instruction embedded during story- 
book reading learned more words than children who did 
not. These results are not surprising as extension activities 
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tend to provide more exposure to words and meanings, 
more opportunities to practice throughout the school day, 
and more contexts with which children are able to connect 
to new words. 

Across studies, extension activities have been designed 
to increase exposure to words and meanings by providing 
(a) repetition of the same instructional activities across 
multiple days (Beck & McKeown, 2007; McKeown & Beck, 
2014); (b) follow-up and review activities that occur after 
book reading (e.g., Coyne et al., 2007); or (c) a variety of 
activities that occur before, during, and after book reading 
(e.g., Neuman, Newman, & Dwyer, 2011; Wasik & Bond, 
2001). Most of these extensions have been designed to 
provide rich opportunities for children to process the 
meanings of new words more deeply (e.g., Coyne et al., 
2007; McKeown & Beck, 2014; Neuman et al., 2011). As 
McKeown and Beck explain, cognitive processing that is 
deeper than rote memory or word associations is critical 
for complete and nuanced word learning. As learners re- 
peatedly engage with and integrate words in a wide range 
of meaningful contexts, rich semantic networks are formed 
and generalization of learning is fostered. 

The purpose of the current study was to evaluate 
Classwide Vocabulary Review Strategies (CVRS), an ex- 
tension to the Story Friends curriculum. This instructional 
component was designed to increase vocabulary learning in 
preschool children receiving the Story Friends small-group 
explicit vocabulary instruction. Teachers were provided 
with prompts and materials to encourage review of and 
opportunities to practice vocabulary words during routine 
classroom activities throughout the school day. The follow- 
ing research questions were addressed: 


1. To what extent does the addition of CVRS to the 
Story Friends program improve vocabulary learning 
compared to the Story Friends program alone for 
children with below-average oral language skills at 
risk for reading difficulties? 


2. To what extent do the CVRS result in vocabulary 
learning among above-average children who are not 
exposed to the small-group vocabulary instruction? 


3. To what extent do teacher social validity results and 
procedural fidelity measures attest to the usability 
and feasibility of Story Friends and CVRS program 
components? 


Method 
Participants 


Four preschool teachers and 23 children were re- 
cruited from four state-subsidized, Voluntary Prekindergarten 
(VPK) classrooms serving children primarily from low- 
income households. The four female classroom teachers 
ranged from 31 to 50 years of age; two were multiracial, 
one was African American, and one was Hispanic. The 
teachers had 3-15 years of teaching experience; each attended 
some college, with one receiving her associate degree. Four 


teachers and 49 parents of children participating provided 
informed consent. One consented child left the preschool 
before pretesting. 

Parents who agreed to allow their children to partici- 
pate were asked to complete a demographic survey about 
their family’s socioeconomic status, the home language 
environment, and the developmental history of their child. 
All children spoke English as their primary language, and 
four of the students spoke an additional language (Spanish, 
Telugu) in their homes. Twenty-two of 23 families completed 
the demographic survey. Characteristics of the children 
enrolled in the study are presented in Table 1. All parents 
reported holding a minimum of a high school diploma 
or General Educational Development, 12 parents held 
a bachelor’s degree (seven mothers, five fathers), and eight 
held a graduate/professional degree (four mothers, four fa- 
thers). Household socioeconomic status (SES) levels were 
classified into five social strata based on Hollingshead’s 
Four Factor Index of Social Status (Hollingshead, 1975), 
as indicated in Table |. The six families with scores below 
30 would be considered of low SES. Two of the 23 children 
had diagnosed disabilities (one with cerebral palsy, the 
other unreported). All participants were eligible to attend 
kindergarten the following year. 

All children whose parents consented were screened 
to determine their eligibility. A total of 48 children from 
four classrooms were initially screened with three screening 
measures: (a) the Picture Naming Subtest of the Individual 
Growth and Development Indicators (IGDI; Wackerle- 
Hollman, McConnell, & Rodriguez, 2017), (b) the Peabody 
Picture Vocabulary Test-Fourth Edition (PPVT-IV; Dunn 
& Dunn, 2007), and (c) the Core Language subtests of the 
Clinical Evaluation of Language Fundamentals Preschool— 
Second Edition (CELF Preschool-2; Wiig, Secord, & Semel, 
2004). These tools are individually administered, norm- 
referenced measures of expressive vocabulary, receptive 
vocabulary, and oral language skills. The IGDI Picture 
Naming subtest was administered to identify children with 
significant language delays (lacking in core vocabulary) and 
those who were exceeding expectations. Thus, children 
with Rausch-based scores between 46 and 52, indicating 
poor to moderate oral language skills relative to a norma- 
tive sample, were identified for further language testing. 
Children were selected if their standard scores were at the 
norm to no more than 1.5 SDs below the mean on either 
the PPVT-IV or the CELF Preschool-2 to be included as 
primary participants. Participants demonstrated mean scores 
on the PPVT-IV, CELF Preschool-2, and IGDI of 93.0 
(SD = 7.2), 90.7 (SD = 6.67), and 48.7 (SD = 2.23), respec- 
tively. Children’s performances on these measures are 
provided in Table |. Three of 26 selected children were 
excluded from the study due to low attendance. 

Ten children with consistent attendance records from 
two of the larger VPK classrooms were initially screened 
for eligibility and excluded due to high PPVT scores (M = 
110, SD = 4.7). These children did not participate in the 
small-group intervention but were exposed to the CVRS. 
By testing these children at posttest, we sought to gain 
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Table 1. Child and family characteristics. 


Class Child Gender Age SES scores* Ethnicity PPVT CELF IGDI_PN 
A 01° F 4-8 46 Asian 101 94 52 
02 F 4-11 53 African American 96 94 47 
03 F 3-11 43.5 African American 97 90 48 
04 F 4-10 42.5 Caucasian 101 102 50 
05 M 4-3 66 Caucasian 97 79 47 
06° F 4-1 66 Asian 101 100 50 
B 07 M 4-0 African American 101 84 47 
08 M 4-7 33 African American 91 90 48 
09° F 441 37 Hispanic/Latino 96 100 50 
10 F 4-6 27 African American 97 86 53 
11 M 4-7 13 African American 100 90 47 
Cc 12 F 5-5 53 Multiracial 85 79 53 
13 F 5-5 53 Multiracial 92 94 
14 M 4-9 53 Multiracial 88 90 48 
15 M 4-9 53 African American 84 84 47 
16 M 5-5 African American 94 100 50 
17 M 4-9 48 African American 84 86 46 
D 18 M 3-10 14 African American 95 86 48 
19 M 4-5 25 African American 76 88 46 
20 F 4-4 28 African American 82 86 48 
21° F 5-0 34.5 Hispanic/Latino 99 100 52 
22 F 4-9 <8 African American 94 94 48 
23 F 4—5 32 African American 87 90 46 
M 92.96 90.70 48.68 
SD 7.12 6.67 2.23 


Note. SES = socioeconomic status; PPVT-IV = Peabody Picture Vocabulary Test-Fourth Edition; CELF = Clinical Evaluation of Language 
Fundamentals Preschool-Second Edition, Core Language Subtest; IGDI_PN = Individual Growth and Development Indicator Picture Naming 


Subtest; F = female; M = male. 


"SES scores were calculated based on Hollingshead’s Four Factor Index of Social Status (Hollingshead, 1975). Scores of 55-66 are in the 
major business and professional strata, scores of 40—54 are in the medium business and minor professional strata, scores of 30-39 are in the 
skilled craftsmen and clerical strata, scores of 20-29 are in the semiskilled workers strata, and scores of 8-19 are in the unskilled workers 
strata. "Families of these children reported that children spoke two languages at home. Child 1 and Child 6 spoke Telegu and English, and 


Child 9 and Child 21 spoke Spanish and English at home. 


preliminary information on the extent of vocabulary learning 
that occurred as a function of the CVRS alone (i.e., with- 
out small-group instruction). 


Setting 


Assessments, screening, and intervention took place 
at four VPK child care centers. These centers were situated 
in high-poverty neighborhoods. The small-group Story 
Friends listening center sessions took place in small quiet 
rooms as designated by school administrators. 


Intervention Components and Materials 


Automated Listening Centers 

The Story Friends program includes automated lis- 
tening centers delivered in small groups and facilitated by 
an adult. Children listen to prerecorded storybooks with 
embedded, explicit lessons to teach challenging vocabulary 
words. Listening center materials include Story Friends 
storybooks, mp3 players with accompanying audio files, an 
audio splitter, and headphones. The storybook set for this 
study included one introductory book (Meet the Jungle 
Friends) and eight instructional books from the Jungle 
Friends book series. The book titles are presented in Table 2. 


The introductory book introduces Jungle Friends charac- 
ters and the procedure of interactive book-reading sessions. 
Each Jungle Friends instructional book includes interac- 
tive embedded lessons to explicitly teach four challenging 
words (e.g., enormous, soar), which would qualify as Tier 2 
vocabulary according to Beck et al. (2013). These vocabu- 
lary words were selected based on their frequency in adult 
language, their utility for academic language instruction, 
their relevance to the storyline, their applicability to a 
variety of contexts, and children’s lack of familiarity with 


Table 2. List of Jungle Friends books used in this study. 


NJF New Jungle Friend 

EFD Ellie’s First Day 

MMA Marquez Monkey Around 
MBD Marquez’s Backwards Day 
EGS Ellie Gets Stuck 

ECF If Elephants Could Fly 
LBF Leo’s Brave Face 

LLR Leo Loses his Roar 


Note. The order of the books was counterbalanced between 
classrooms. 
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the target words (based on prior testing with preschoolers). 
Each target vocabulary word is taught in two embedded 
lessons per book that include systematic instructional 
language. During each lesson, children are provided with 
three opportunities to say the word, two opportunities to 
say the definition, an example of the word in the context 
of the story, an example of the word in a common life 
experience, and an activity related to the word’s meaning. 
The children respond to the book narrator and speak out 
loud while wearing headphones; the adult monitoring 
looks on expectantly and prompts children on occasion. 
Children listen to each book three times per week. Each 
Story Friends prerecorded audio includes four target 
vocabulary words that are 10-12 min in length. In total, 
children spent less than 40 min in small groups each week. 


CVRS 

The purpose of CVRS is to prompt teachers to pro- 
vide opportunities to review and practice the use of target 
vocabulary words in contexts beyond their use in Story 
Friends books. The materials are designed to be easy to 
use, flexible, and readily incorporated into a wide range of 
typical classroom routines. 

As part of the CVRS condition, teachers were prompted 
to review target words with text message reminders and 
in-class visual reminders and materials. Daily text message 
reminders modeled five sentences per week that could be 
used in various classroom conversations. These sentences 
included phrases that related to children’s experiences, or 
that offered further explanations of words, or prompts for 
children to complete sentences or respond to requests or 
questions (e.g., “You are birds! Lift up your wings and Soar 
to the door!’). These text messages were sent to the teachers 
at their preferred time of the day. Visual reminders and 
materials included word cards, a weekly word chart, and a 
review board. The word cards were designed to remind 
teachers to use phrases with target vocabulary words during 
school time. The weekly word chart included Velcro-backed 
cards with the Tier 2 words and meanings to remind 
teachers to use the target words and help them keep track 
of their frequency of word use. The review board functioned 
as a word wall to prompt review of previously taught target 
vocabulary words and included small pictures from the 
story as well as real-life pictures (see Figure 1). 


Procedure 


The duration of the study from teacher training 
through the administration of maintenance probes was 
11 weeks. Prior to implementation, classroom teachers 
received a brief training (~60 min) in a quiet room to 
discuss the program procedure and expectations. The follow- 
ing topics were presented at the meeting: (a) the benefits 
of word learning on children’s academic success, (b) sched- 
uling and situating Story Friends listening centers for 
three sessions each week, (c) the rationale of Story Friends 
CVRS and how to make use of the materials provided, 
and (d) the daily expectations, requirements, and roles for 


research staff and teachers during the study. At the end of 
training, research staff shared the classroom extension 
materials and helped teachers set them up in the classroom. 


Story Friends Listening Centers 

Trained research staff facilitated the Story Friends 
listening centers three times a week. The role of the facili- 
tator was to ensure that each child had a headphone and 
a book and to encourage children to participate (e.g., 
responding to the narrator’s instructions) and to stay on 
task (e.g., turning the pages). The facilitator provided praise 
for on-task behavior and redirected children when neces- 
sary but did not provide any additional instruction or 
explicit approval of the children’s responses. Implementa- 
tion of listening centers by research staff ensured that the 
teachers were blind to words introduced in the books in 
the Story Friends—only control condition. 

Children participated in the listening centers in 
groups of three or four with the facilitator. Prior to listen- 
ing to any of the instructional books, participants listened 
to the Story Friends introductory book (Meet the Jungle 
Friends) one time. The introductory session enabled them 
to practice listening center expectations and procedures, 
such as responding to the interactive lessons and turning 
pages when directed. 

In both conditions, children participated in small- 
group centers three times per week. Prompts and materials 
used in the CVRS condition were introduced every other 
week. During the CVRS condition, teachers practiced target 
vocabulary words from a given book during the week the 
book was introduced, and they continued to practice the 
same vocabulary words for a second week as children lis- 
tened to a new book in the listening center—only condition. 


Dependent Variables 


Children’s knowledge of targeted vocabulary words 
was assessed using a mastery monitoring probe. Mastery 
monitoring vocabulary probes intermixed testing of words 
to be taught with words that were previously taught. Words 
were each assessed at four time points: pretest (before 
listening to storybooks), posttest (after completing three 
listening centers), Maintenance | (1 week after posttest), 
and Maintenance 2 (2 weeks after posttest). Thus, testing 
occurred on four successive weeks for each book. During 
the mastery monitor probes, children were asked to pro- 
vide definitions for targeted vocabulary words (e.g., 
What does soar mean?). The child responses were scored 
as reflecting complete knowledge (2 points), partial knowl- 
edge (1 point), and no knowledge (0 points), with a 
maximum score of 8 for four target vocabulary words per 
book. 

Social validity assessments were conducted to evaluate 
self-reported teacher perceptions regarding their impressions 
and satisfaction with the program and its feasibility. The 
two assessments included a structured interview with seven 
open-ended questions and a survey with nine Likert-scale 
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Figure 1. Components of Classwide Vocabulary Review Strategies. 


Instructional 
component 


Delivery method 


Example 


Word Cards Teachers received the 
review cards weekly. 
These cards provided target 
vocabulary words, their 
meanings, and example 


phrases. 


Weekly Word 
Chart 


Teachers provided 
opportunities and review 
vocabulary words 
throughout the school day 
and placed cards when 
used. 


Story Friends 
Review Board 


Teacher placed cards in 
board at the beginning of 
week that followed the 
treatment book. 


questions that evaluated teachers’ attitudes and ideas re- 
garding the Story Friends curriculum. 

Research staff who facilitated the listening centers 
collected field notes during school visits that overlapped with 
classrooms’ circle and center times. These field notes were 
not collected systematically, but we took advantage of eight 
20-min opportunities to gather examples of how teachers 
were addressing target words in the classroom. The observers 
transcribed these teacher—child conversations verbatim. 


Fidelity of Assessment Administration 
and Scoring Reliability 


The research staff received training on all mea- 
sures. This training covered administration of the CELF 
Preschool-2, PPVT-IV, and IGDI Picture Naming stan- 
dardized assessment tools, as well as the researcher-developed 
mastery monitor vocabulary probes. Mastery monitor vo- 
cabulary assessment sessions were recorded to examine 
the fidelity of test administration. A second research staff 
member listened to 20% of all testing sessions and judged 
whether items were read correctly, responses were recorded 
accurately, and the standard prompting protocol was 
followed. Fidelity of assessment administration averaged 
89% (range: 71%-98%). 

After a primary trained researcher scored all mea- 
sures, a second trained staff member rescored 20% of the 
mastery monitor vocabulary assessments to assess reliabil- 
ity. Item-by-item response agreement was calculated by 
dividing the agreements by the total number of agreement 
and disagreements. Interrater agreement percentages were 
high, averaging 96.9% (range: 93.75%-100): 100% at 


pretest, 100% at posttest, 93.75% at Maintenance 1, and 
96.87% at Maintenance 2 probes. 


Experimental Design and Analysis 


As in prior evaluations of Story Friends intervention, 
a repeated acquisition design was used to determine the 
extent to which vocabulary learning effects were replicated 
within and across participants for each book. The crite- 
rion for demonstrating experimental effects required a gain 
from before intervention to after intervention of at least 
2 word points (e.g., the equivalent of learning one word or 
demonstrating partial knowledge of two words) for each 
book. Thus, we calculated the percentage of books for which 
learning effects were replicated. 

To answer our primary research question for this 
study, an adapted alternating treatments design (AATD) 
was used to compare the effects of Story Friends listening 
center instruction alone (SF) versus Story Friends listening 
center instruction with CVRS. The AATD was selected 
because of its applicability to comparing the effects of 
instructional practices with nonreversible behaviors (Gast & 
Ledford, 2014). The order of the two conditions was coun- 
terbalanced across classrooms to control for possible differ- 
ences in word difficulty among books. The SF condition 
included three listening center sessions and lasted 1 week 
for each book. The CVRS condition also included listen- 
ing centers lasting 1 week and added teacher-implemented 
classroom vocabulary review strategies that occurred for 
2 weeks. These two conditions were initiated in an alternat- 
ing fashion with each new book, and book assignment was 
counterbalanced among classrooms. Thus, four books were 
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assigned to the SF condition, and four books were assigned 
to the CVRS condition, except for Class B, which with- 
drew from the study after six books because the classroom 
teacher transferred to another preschool. 

Visual analyses, descriptive statistics, multilevel model- 
ing procedures, and summaries of social validity and pro- 
cedural fidelity measures were used to evaluate treatment 
effects. First, visual analysis allowed researchers to deter- 
mine whether functional relations between treatments and 
vocabulary learning were demonstrated, whether learning 
was enhanced by CVRS, and the extent to which results 
were replicated within and across participants (Gast & 
Ledford, 2014). 

Second, vocabulary gains were computed for each 
student for each of the eight books by subtracting pretest 
scores from the average of vocabulary scores for the post- 
test, Maintenance 1, and Maintenance 2. These gain scores 
were initially analyzed descriptively by computing least 
square estimates, that is, mean gain across books for each 
experimental condition (SF and CVRS) for each participant. 

Third, a two-level, linear, cross-classified, random 
effects modeling procedure was used to further character- 
ize variations in children’s vocabulary gains. Although the 
automated nature of SF listening centers results in high 
treatment fidelity, teachers are more likely to vary in the 
extent to which they implement CVRS. Because children 
are nested in classrooms, multilevel modeling was applied. 
More specifically, fixed effects in the model allowed us to 
estimate and test the average difference in gains between 
the two treatment conditions (CVRS and SF), as well as to 
estimate differences in gains between the classrooms, which 
we treated as fixed effects because there were less than five 
classrooms, and to test the classroom by treatment effects. 
The random effects in the model allowed us to examine 
the variation in the gains across books and children. The 
cross-classified random effects modeling analyses were con- 
ducted using SPSS mixed models with restricted maximum 
likelihood estimation. Satterthwaite estimated degrees of 
freedom were applied to obtain fixed-effect inferences that 
have been shown to be accurate with single-case data 
(Ferron, Bell, Hess, Rendina-Gobioff, & Hibbard, 2009; 
Ferron, Farmer, & Owens, 2010). 

Finally, social validity, procedural fidelity, field 
notes, and teacher-reported implementation data provided 
rich data sources to evaluate the teacher perceptions and to 
gather examples of teacher-improvised implementation. 
After compiling and organizing the qualitative data from 
various resources, the common themes were identified 
through open coding and code condensation (Creswell & 
Poth, 2017). 


Results 
Comparing Vocabulary Gains Across Conditions 


Visual analysis of the immediacy, level, variability, 
and maintenance of effects was conducted for 23 graphs 
presented in Figures 2-4. Data points reflect correct responses 


for four target vocabulary words per book before inter- 
vention (pretest), after intervention (posttest), and after the 
second and third weeks (Maintenance | and 2). Thus, vo- 
cabulary learning scores were plotted at four time points 
for each book, with closed triangles denoting the Story 
Friends—only (SF) condition and closed circles denoting 
the Story Friends + CVRS condition. The expected learning 
trend was an abrupt change in level after introducing the 
SF intervention. However, higher levels and greater stabil- 
ity were expected for the SF + CVRS condition. 

First, using the repeated acquisition design, we evalu- 
ated whether there were repeated improvements from 
pretest to posttests/maintenance tests as books were intro- 
duced each week. The 23 children were exposed to as many 
as eight books each, resulting in a total of 174 possible 
replications. Learning effects of an increase of at least 2 
word points was demonstrated for 89% of the 174 possible 
replications. Because there were 3 weeks of posttesting and 
maintenance testing for each book, variability in perfor- 
mance can be seen (see Figures 2-4). Most often, the per- 
formance stayed relatively consistent with the posttest. 
You will see examples of drop-offs, stability, and increases 
in learning gains in both the SF and CVRS conditions, 
with more examples of continued growth in the CVRS 
condition. Interestingly, a lack of learning effects was dem- 
onstrated 17 times for books during the SF condition and 
only three times during the CVRS condition. 

Second, the first and last authors independently 
judged whether the AATD revealed differential experimen- 
tal effects for each participant when comparing the SF and 
CVRS conditions. They were in agreement for 91% of the 
participants. Figures 2-4 are organized according to the 
agreed-upon results of this visual analysis. Figure 2 pre- 
sents 12 children who demonstrated overall higher perfor- 
mance in the CVRS condition than in the SF condition; 
Figure 3 presents three children who did not show differen- 
tial effects because of ceiling effects (i.e., very high vocabu- 
lary learning in both conditions); and Figure 4 presents 
eight children, six of whom showed inconsistent differential 
effects between conditions and two children (P18 and P19) 
who showed weak learning effects in general. 

We averaged the gains for each of the books assigned 
to each condition to obtain the mean gain for each child 
(see Table 3). Average child vocabulary gains in the CVRS 
condition were 4.7 word points (SD = 1.37) versus a mean 
of 2.8 word points (SD = 1.16) in the SF condition, a dif- 
ference of 1.9 word points. Average child vocabulary gains 
also varied among classrooms. As can be seen in Table 3, 
Classrooms A, B, C, and D showed differences in mean 
gain scores between the CVRS and SF conditions of 0.6, 
2.5, 1.7, and 3.0 word points, respectively. Based on visual 
inspection of the AATD, we identified 12 children who 
demonstrated higher performance in the CVRS condition; 
their average gain over the SF condition was 2.86 word 
points. Interclass correlation coefficient variance proportions 
were calculated for the unconditional model. The interclass 
correlation coefficients in this analysis were .31 for children 
and .049 for books. Hence, a two-level, linear, cross-classified, 
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Figure 2. Results of repeated acquisition and adapted alternating treatments designs showing superior word learning for the Classwide 
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Figure 3. Results of repeated acquisition and adapted alternating treatments designs showing ceiling effects for the Classwide Vocabulary 
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Table 3. Means and standard deviations of child vocabulary gains by experimental condition. 


Story Friends plus Classwide 
Vocabulary Review Strategies 


Sn Eeaeny Mean difference 


Child ID M SD M SD between conditions 
01° 5.50 1.66 6.25 0.43 -0.75 
02° 5.67 1.51 2.25 0.72 3.42 
03° 3.33 1.20 2.92 1.72 0.41 
04? 4.33 1.39 5.50 1.50 -1.17 
05° 4.75 1.11 4.58 2.19 0.17 
06° 5.58 1.42 4.25 1.82 1.33 

Classroom A 4.86 1.38 4.29 1.40 0.57 
07° 4.89 0.31 2.78 1.34 2.11 
08° 2.67 1.41 1.00 0.82 1.67 
og? 6.33 2.81 2.44 2.20 3.89 
10 5.56 2.64 2.89 0.83 2.67 
11° 4.11 1.99 2.00 1.63 2.11 

Classroom B 4.71 1.83 2.22 1.36 2.49 
12° 7.00 1.73 4.42 1.09 2.58 
137 6.17 1.28 5.75 1.38 0.42 
14° 3.42 1.21 2.25 0.64 1.17 
15° 3.67 0.33 1.58 0.98 2.09 
16° 5.50 1.66 4.08 1.71 1.42 
17° 3.50 1.09 0.83 0.87 2.67 

Classroom C 4.88 1.22 3.15 7.11 1.73 
18° 1.71 1.14 0.83 1.26 0.88 
19° 1.92 1.92 0.42 0.28 1.50 
20° 3.75 1.75 0.75 0.36 3.00 
21 6.75 0.60 3.17 1.21 3.58 
22° 6.67 0.91 3.58 1.14 3.09 
23° 6.33 0.53 0.50 0.55 5.83 

Classroom D 4.52 1.14 1.54 0.80 2.98 

Grand mean 4.74 1.37 2.83 1.16 1.92 


Note. Italicized values indicate summary parameters for each classroom. 


*Children who demonstrated high vocabulary knowledge in both conditions. The range of mean difference scores of conditions are -1.17 to 
0.42, averaging —0.50. °Children who demonstrated overall higher vocabulary knowledge in the Classwide Vocabulary Review Strategies 
condition than the Story Friends—only condition. The range of mean difference scores of conditions are 1.33-5.83, averaging 2.86. 
°Children who demonstrated inconsistent performance between conditions. The range of mean difference scores of conditions are 0.17-3.08, 


averaging 1.40. 


the differences in this gain for Classes A, B, and C are 
Yo1, You, and Yo3, respectively. The difference in average 
gain between the CVRS and SF conditions for Class D 

is Yio, whereas Classes A, B, and C differ from Class D 
in the effect of CVRS relative to SF by yy, Y12, and y;3, 
respectively. The error terms (1.€., rijx, Vox» Uojs Vik, and u4;) 
are assumed to be normally distributed. 

The proposed model statistically fits better than an 
unconditional model, y* = (11, N = 174) = 161.8, p < .01. 
In addition, the model assumptions were evaluated by 
examining each of the residuals in the model using a box 
plot for outliers, a scatter plot for homoscedasticity, and a 
QQ-plot for normality. Results indicated a homoscedastic 
residual distribution with multivariate normality and inde- 
pendence across levels and that observations were indepen- 
dent conditional on the variance components. 


As shown in Table 4, the estimated parameter repre- 
senting child vocabulary gains for the SF condition aver- 
aged 1.50 word points for the reference classroom (i.e., 
Classroom D). For Classrooms A, B, and C, the average 
gains were estimated to be 2.60, 0.81, and 1.49 word points 
higher, respectively. Thus, the average gains for the four 
classrooms for the SF condition were 4.10, 2.31, 2.99, and 
1.50 word points. The classroom extension condition 
expanded child vocabulary gains by an estimated 2.80 word 
points for Classroom D, which was a statistically signifi- 
cant difference, (24) = 4.75, p < .001. When factoring in 
the Class x Treatment interactions, the average gains for 
the CVRS condition were 4.80, 4.80, 4.96, and 4.30 for 
Classrooms A, B, C, and D, respectively. 

As seen in Table 4, the model also revealed several 
statistically significant variance components. Namely, the 
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Table 4. Output of full-model mixed effects for estimating child vocabulary gains. 


Parameter Parameter estimate SE 95% Cl Significance 
Fixed effects 
Intercept 1.502 0.649 [0.16, 2.85] .030 
Class A 2.596 0.880 [0.76, 4.43] .008 
Class B 0.805 0.932 [-1.32, 2.74] .397 
Class C 1.486 0.887 [-0.36, 3.33] 109 
Class D _— _ _ _— 
Treatment 2.802 0.590 [1.58, 4.02] .000 
Class A x Treatment -2.102 -0.792 [-3.74, -0.45] .014 
Class B x Treatment -0.310 0.845 [-2.54, 1.43] 117 
Class C x Treatment 0.834 0.810 [-2.51, 0.83] 313 
Class D x Treatment _ — _ _— 
Variance estimates 
Within child 2.828 0.189 (2.48, 3.22] .000 
Between child 
Intercept 1.997 0.728 [0.97, 4.08] .006 
Treatment effect 1.213 0.562 [0.49, 3.01] -031 
Covariance 0.316 0.466 [-1.23, 0.60] 498 
Between books 
Intercept 0.247 0.195 [0.53, 1.16] .206 
Treatment effect 0.237 0.265 [0.27, 2.11] .370 
Covariance -1.672 0.197 [-0.55, 0.22] 396 


Note. Cl = confidence interval. Dashes represent the “reference group” in the analyses. 


residual variance within children (o” = 2.83), the variance 
between children for the SF condition (ty9 = 2.00), and 
the variance between children for the treatment effect 
(Ty = 1.21), that is, the difference between the gains in the 
CVRS and SF conditions, were significant. The random 
variance associated with the books was not statistically 
significant. 


Learning of Children Exposed to Words Through 
Classwide Vocabulary Review Activities 


CVRS occurred in a classroom context, which allowed 
all children to hear targeted words, definitions, and teach- 
ing examples. Although the listening centers were run by 
the research assistants, classroom teachers were responsible 
for reviewing the words in the CVRS condition. Thus, 
learning opportunities varied as a function of the extent to 
which teachers implemented practice activities with children 
in their classrooms. Upon completion of the study, we 
analyzed the vocabulary learning of 10 children who did 
not receive listening centers from Classrooms C and D 
by asking them to define vocabulary words (mastery moni- 
tor probes) from both conditions. The maximum vocabu- 
lary score was 32 word points per condition (4 books x 
4 target words x 2 points per word). Results are shown in 
Figure 5. The means of children’s vocabulary scores for 
the SF condition words versus for the CVRS condition 
words were 5.8 (SD = 3.9) and 19.8 (SD = 5.2), repre- 
senting 18.1% and 61.9% correct, respectively. A paired- 
samples ¢ test indicated significantly better vocabulary 
knowledge for the CVRS condition, (9) = 6.89, p < .001, 
d=2.18. 


Teacher Implementation and Social Validity 


The four teachers who participated in the study 
showed various frequencies and forms of targeted 
vocabulary use during daily classroom instruction. For 
lack of direct observational data, we used three sources 
of information to characterize the extent to which 
teachers taught targeted vocabulary words during class- 
room activities: word cards placed on Story Friends 
Weekly Word Charts and Review Boards, field notes 
collected during school visits, and social validity sur- 
veys and interviews conducted with the teachers at the 
end of the study. For example, field notes included 
the examples of teacher—child dialogues with target 


Figure 5. Ten typical children’s correct definitions of classroom 
exposure words versus nonexposure words at posttest (without 
the Story Friends listening center intervention). CVRS = Classwide 
Vocabulary Review Strategies. 
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vocabulary words. An example of such an exchange 
follows: 


Child: I was sick. 

Teacher: You were ‘ill’. What did your mom do to 
make you feel better? 

Child: Medicine. 

Teacher: Did she ‘comfort’ you? 

Child: yes. 


After each vocabulary use, teachers were asked to 
add a word card to the word chart. We did not observe 
teachers doing so with great regularity. Teacher-reported 
frequency averaged 5.8 word cards per week (SD = 2.05). 
The teacher-reported frequencies of vocabulary exposure 
averaged 5.5 word cards per week (SD = 0.89) for Class- 
room A, 8.5 (SD = 2.2) for Classroom B, 5.9 (SD = 0.44) 
for Classroom C, and 3.5 (SD = 1.2) for Classroom D. 
Although this underestimates actual extension activities, it 
seemed to reflect relative frequency of engagement in such 
practices among classrooms. 

Analysis of the social validity survey (nine Likert- 
scale questions with a range of 1-4) indicated that teachers 
were motivated to implement the program to support vo- 
cabulary learning in their classrooms (M = 3.75). Teachers 
strongly agreed that they received explicit program imple- 
mentation instruction (M = 4.0) and understood how to 
use the program (M = 4.0). They expressed that the amount 
of time required to implement this program was reasonable 
(M = 3.75) and the program implementation required sup- 
port from coworkers (M = 3.25). Teachers also reported 
that they would be excited to use this program (M = 3.75) 
and the program can be implemented as frequently as 
desired (MM = 3.5). They all reported that their frequency of 
self-reported vocabulary use was an underestimation of their 
actual frequency of vocabulary use. Teachers mentioned 
the challenges of adding cards to the classroom chart after 
each vocabulary use, such as during outside time. Thus, the 
implementation frequency reported should be evaluated 
cautiously. 

Five topics emerged based on the results of teacher 
social validity interviews, which will be discussed 
below. 


Overall Vocabulary Use in the Classroom 

Teachers reported three strategies for embedding 
targeted vocabulary into daily conversations: (a) Teachers 
taught words to the whole group during circle time, 
(b) teachers noticed when children used the words and 
followed up with prompts for other children, and (c) teachers 
targeted words that fit easily into suitable contexts (e.g., 
“when the kids ride their bicycle, I say ok, hold on, you can 
collide with your friends. The other is destroy, when you 
build the blocks, your friends may destroy it.”). 


Observed Child Reactions Toward the Program 
and Target Vocabulary Word 

Teachers expressed that the children were curious 
and excited about the program and target words. Children 


used them in the conversations with their classmates, asked 
the meanings when they could not recall, and pointed out 
their meanings without the teacher asking (e.g., “They kind 
of go with it and if I say ‘leap’, somebody will yell ‘to jump’ 
instead of me asking for the meaning.”’). 


Evaluation of Classroom Chart and Review Board 

Teachers noted that the Story Friends classroom 
visuals were constant reminders of words to use repeatedly 
during a day. One teacher stated that the visuals helped 
her to keep track of the less frequently used words so that 
she could use these words more frequently. 


Recommended Example Sentences and Daily Text Messages 

Overall, teachers said that they utilized the example 
sentences and their own sentences. One teacher expressed 
her appreciation for receiving various sentence structures 
recommended by the research team (i.e., “J liked how you 
guys taught me how to use them with the kids.”). 


Overall Teacher Reaction to the Program 

Teachers evaluated what they liked about CVRS of 
the Story Friends program: “The program gave different 
vocabulary words that I usually didn’t use in the class- 
room.” “I like it a lot. Children were excited about the 
books, new words, and headphones.” “I like the words 
and how they were presented.” and “The program was well 
put together and very organized, materials that were pro- 
vided, text messages, books, word wall, listening center, 
and small group time.” 


Discussion 


Results indicated that the Story Friends program 
with classroom extensions produced child vocabulary gains 
approximately twice as much as Story Friends alone. This 
improvement in learning was functionally related to the 
CVRS condition in a predictable way. First, 23 preschool 
children from four classrooms increased vocabulary learn- 
ing by at least 2 word points during the CVRS condition 
for 84 books of the 87 possible replications (97%) in com- 
parison to 70 books of 87 possible replications (80%) during 
the SF condition. Second, the Story Friends program with 
CVRS significantly enhanced child vocabulary knowledge 
compared to the SF condition. Based on posttest scores, 
children’s demonstrated vocabulary knowledge averaged 
41.9% of words in the SF condition. The same children 
demonstrated better learning, averaging 62.3% of vocabulary 
words taught during the CVRS condition. The percentage 
of the vocabulary learning in the SF condition was in 
line with previous Story Friends findings (Goldstein et al., 
2016; Greenwood et al., 2016; Kelley et al., 2015; Spencer, 
Goldstein, Sherman, et al., 2012). 

Adding CVRS to the Story Friends program sub- 
stantially improved child vocabulary learning compared to 
previous studies (Goldstein et al., 2016; Kelley & Goldstein, 
2014; Kelley et al., 2015). The 20.4% increase shown in this 
study reflects a marked improvement. This result extends 
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the literature demonstrating positive effects of increased 
opportunities to practice vocabulary words throughout the 
school day beyond the instruction in storybook contexts 
(Coyne et al., 2007, 2009; Loftus-Rattan et al., 2016; 
McKeown & Beck, 2014; Wasik & Bond, 2001). Compar- 
ing studies is complicated by the differences in the number 
and difficulty of words taught and differences in the mea- 
sures of word learning. Investigators have suggested that 
adults can support child vocabulary learning by initiating 
multiple conversations with explicit instruction using target 
vocabulary words (Loftus-Rattan et al., 2016; Silverman 
& Crandell, 2010). The CVRS condition exemplifies a 
viable means of prompting and promoting explicit vocabu- 
lary instruction through such adult-child conversations. 
Indeed, teachers reported that they benefited from text 
messages and visual materials that reminded them to embed 
target words into daily classroom interactions. 

One conceivable explanation of increased child 
vocabulary knowledge could be multiple explicit vocabu- 
lary instructional episodes in meaningful contexts to which 
children could relate (Beck et al., 2002; Marulis & Neuman, 
2010; Spencer, Goldstein, & Kaminski, 2012). Children 
were provided ample opportunities to respond, which is a 
robust predictor of learning (Greenwood, Delquadri, & 
Hall, 1984; Greenwood, Horton, & Utley, 2002; Sutherland 
& Wehby, 2001). Researchers have posited that child vo- 
cabulary learning progresses in stages, for example, from 
recognizing words, to comprehending words, to expressing 
words, to defining words (Beck et al., 2013; Christ, 2011; 
Goldstein et al., 2017; McKeown & Beck, 2003). Our study 
exemplifies review strategies that are feasible for implemen- 
tation in various contexts during classroom routines to 
provide multiple opportunities to learn and practice novel 
vocabulary words. 

The combination of repeated acquisition design and 
AATD presented unique opportunities to apply multiple 
data analytic techniques. The repeated acquisition design 
essentially replicated effects consistent with previous evalu- 
ations of the SF intervention mainly with preschoolers 
from low-income communities. AATDs are among the few 
options for comparing two (or more) treatment conditions 
using single-case experimental designs. However, they are 
risky designs because they are not very sensitive to detect 
small differences. Moreover, this potential problem tends 
to be exacerbated when including a treatment repeatedly 
shown to result in substantial learning. Indeed, ceiling effects 
masked potential treatment differences for at least three 
participants, which was consistent with high achievers in 
past studies of Story Friends. 

Although it is unusual to include 23 participants in 
single-case design experiments, it allows one to examine 
individual differences in learning not captured in group 
design experiments. A novel and notable aspect of this 
study was the use of multilevel modeling analysis. One 
advantage was the use of multilevel modeling to charac- 
terize the individual differences among classrooms, largely 
attributable to differences in teacher implementation, as 
well as other potential sources of variability, for example, 


books. This analysis also has the advantage of providing 
an overall estimate of the magnitude of our treatment 
effect, expressed in word point gains. Although these effects 
and the differences between classrooms were explored 
descriptively in Table 3, the use of the model provided a 
formal mechanism to estimate the effects, express uncer- 
tainty in the effect estimates, and examine moderators. 
That is, the point estimates from the model summarized 
and mirrored the descriptive mean differences provided in 
Table 3, but by using the model, we obtained standard 
errors that accounted for the complex cross-classified struc- 
ture of the data. These standard errors facilitated the crea- 
tion of confidence intervals and hypothesis tests for the 
effects, as well as tests for the differences in effect between 
classrooms. 

Despite the large effects revealed through statistical 
analyses, significant variation in vocabulary learning was 
evident among children. Our visual analyses revealed dif- 
ferent learning patterns that influenced our comparison of 
the CVRS and SF conditions: (a) Three children demonstrated 
near-maximal learning in both conditions, (b) 12 children 
demonstrated consistently better learning outcomes during 
the CVRS condition, (c) six children demonstrated incon- 
sistent differences in vocabulary learning across conditions, 
and (d) two children showed minimal learning in both 
conditions. Visual inspection may result in rather conserva- 
tive judgments of experimental effects. For example, Child 
12 had the highest overall gains associated with the CVRS 
condition, but SF gains were almost as high for two of four 
books, which resulted in our judgment of inconsistent 
effects. As can be seen in Figure 4, even among the 26% of 
children who did not show a consistent advantage of the 
CVRS over the SF condition, an impressive amount of 
word learning was demonstrated. 

These results shed light on how preschool children 
respond differently to vocabulary intervention programs 
and may inform strategies for moving children among tiers 
of instruction to maximize learning. Certainly, we would 
consider the two children who demonstrated minimal 
learning in both conditions to be candidates for a higher 
tier of instruction. We realize that children with limited 
language skills may be at a disadvantage in learning chal- 
lenging vocabulary words if they have trouble understand- 
ing the definitions or contexts used to illustrate the use of 
these novel words. Some children could learn and retain 
new words and their meanings more rapidly than other 
children. Others may have memory, attention, or behavior 
problems that interfere with learning. Consequently, we 
explored whether results were predicted by PPVT and 
CELF scores. The only significant correlation was between 
PPVT and the SF condition (r = .52, p < .05). Child 19, 
with the lowest PPVT score (76), was one of the two chil- 
dren demonstrating minimal learning, albeit slightly better 
than Child 18 who had a PPVT score of 95. Notably, 
these two children were among the six children from the 
low social strata. However, most of the children from low- 
income homes demonstrated good word learning, especially 
in the CVRS condition. We found no readily apparent 
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pattern based on standardized pretest scores. In general, 
children with lower vocabulary and language test scores 
seemed to benefit quite a bit from the CVRS condition. 
Future research is needed to better understand the sources 
of variation in vocabulary learning among young children 
with various background knowledge and learning trajecto- 
ries and how to overcome poor learning outcomes. Never- 
theless, results of this intervention show promise for 
narrowing the alarming word gap often found among chil- 
dren living in impoverished homes and communities. 

The classroom extension activities also presented an 
opportunity to observe vocabulary learning among chil- 
dren who were excluded from small-group sessions due to 
their higher PPVT scores. These children were exposed to 
vocabulary words and definitions only in the CVRS condi- 
tion during classroom routines. Upon the completion of 
the program, these children learned a mean of 61.9% of 
CVRS condition vocabulary words compared to a mean 
of 18.1% words to which they were not exposed. The base 
rate of 18.1% is slightly lower than the pretest mean (21.9%) 
for the three participants who showed ceiling effects. The 
latter participants showed even more improvement averag- 
ing 90.8% of words known at posttest across the two con- 
ditions. Nevertheless, the average vocabulary knowledge 
of these children as a function of CVRS alone was as much 
as the average vocabulary knowledge (62.3%) of children 
with limited language skills who participated in SF listen- 
ing centers + CVRS condition. Hence, teacher imple- 
mentation of CVRS could help elevate the vocabulary 
knowledge of children with typical and limited language 
skills. The typical children are able to increase their 
vocabulary knowledge by listening and observing teachers’ 
language use during large group instruction (Marulis & 
Neuman, 2010). Thus, teachers may efficiently provide 
instruction at the general classroom and small group 
levels. 

We also sought to judge the acceptability and feasi- 
bility of asking teachers to extend review and practice into 
their classrooms. Because teachers implemented CVRS at 
various times across the school day, and capturing accurate 
data on implementation rates was not possible, we relied 
on multiple, subjective sources of information from teachers 
and research staff. Social validity measures indicated that 
teachers were able to implement classroom extensions of 
the Story Friends curriculum, and they perceived the strat- 
egies to be engaging and beneficial to young children. 
Despite the current literature pointing to the need to pro- 
vide more teacher training and coaching for better child 
vocabulary learning (Hindman & Wasik, 2012; Neuman & 
Wright, 2010), the present intervention only required a 1-hr 
teacher training to introduce the program. Thereafter, 
daily text messages and classroom visual materials were 
provided to remind teachers to use the target vocabulary 
words. 

Field notes from research staff indicated that teachers 
incorporated provided materials in a number of ways. They 
implemented large group instruction during circle time, 
used target words instead of common words during daily 


classroom instruction, and capitalized on teaching opportu- 
nities when children used the words. In contrast to prior 
research, this study demonstrated that teachers can reinforce 
the learning in structured vocabulary interventions with 
minimal training (Dickinson, 2011; Justice et al., 2008). 
Some of these studies describe the low likelihood of finding 
robust vocabulary instruction in low-SES classrooms with 
diminished classroom and teacher resources (Dickinson, 
2011; Wright, 2012). Perhaps, the focus on relatively few 
targeted words made this a more achievable goal for teachers. 
Although we do not know how many review opportunities 
were provided, results revealed benefits to vocabulary 
learning for a majority of the children in classrooms. 

Observations of research staff indicated that teachers 
varied in the extent to which teachers extended vocabulary 
instruction into daily classroom routines. Staff members 
noted that the teachers were not very good at documenting 
review opportunities on the chart that was provided for 
that purpose. Nevertheless, they agreed that the relative 
ranking of teacher implementation was consistent with the 
number of words added to word wall charts. Future re- 
search needs to capture perceptions of implementation 
more systematically to relate class differences to differences 
in vocabulary learning. 

How to best monitor teacher implementation of 
classroom practice activities needs to be determined. This 
is key to developing an understanding of how to prompt 
and reinforce classwide vocabulary practice needed to 
optimize results. This information would provide a better 
understanding of how the current intervention program 
should be implemented in preschool settings by classroom 
teachers and instructional aides. Our results on teacher us- 
age of target words underestimated implementation. Other 
self-monitoring strategies might be more successful (Kalis, 
Vannest, & Parker, 2007; Oliver, Wehby, & Nelson, 2015). 
Future work must devise alternatives to expensive, time- 
consuming observations across the day to assess the fre- 
quency of teacher instruction targeting vocabulary during 
classroom routines. It may be beneficial to incorporate a 
recording system that captures teacher utterances, such as 
LENA Pro. We also need to figure out how to best incor- 
porate practice at home to optimize the effects of vocabu- 
lary instruction. 

Another limitation of this study is that the Story 
Friends small-group sessions were implemented by the 
researchers and not educational staff. This likely helped 
us maintain experimental control as teachers likely were 
unaware of words on alternate weeks and thus less likely 
to implement CVRS in the SF condition. Nevertheless, the 
Story Friends program was designed to provide an auto- 
mated curriculum delivered to small groups of children by 
any educational staff (e.g., instructional aides). Previous 
studies demonstrated high fidelity of the classroom imple- 
mentation of small-group sessions, as a prerecorded curric- 
ulum with well-constructed explicit instruction does not 
require intensive training (Goldstein et al., 2016; Greenwood 
et al., 2016; Kelley & Goldstein, 2014; Kelley et al., 2015). 
Teaching staff merely need to provide minimal monitoring 
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and input, such as helping children to turn pages and en- 
couraging children to stay on task. Moreover, teachers may 
benefit from knowing contexts used when vocabulary words 
are introduced and how they are taught. Thus, we believe 
that asking teaching staff to implement daily listening cen- 
ters and classwide review activities is likely to prove viable. 

In conclusion, the purpose of this study was to exam- 
ine the effects of adding classroom review and practice to 
the Story Friends program on the learning of challenging 
vocabulary among preschoolers with limited oral language 
skills. Findings indicated significant improvements in chil- 
dren’s learning of targeted vocabulary. Additionally, we 
provided preliminary evidence of how children with above- 
average language scores can advance their target vocabu- 
lary knowledge merely as a result of the classroom review 
activities. Classroom extensions of the Story Friends pro- 
gram appear to be feasible to implement in preschool edu- 
cation settings and a promising approach to improving 
children’s target vocabulary knowledge. 
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